Papers with augmentation methods
AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation suffer from potential semantic damage due to the discrete nature of sentences. |
| Approach: | They propose to adapt AutoAugment to solve this problem by using softEDA to increase text data. |
| Outcome: | The proposed method can boost existing augmentation methods and enhance cutting-edge pretrained language models. |
DialAug: Mixing up Dialogue Contexts in Contrastive Learning for Robust Conversational Modeling (2022.coling-1)
Copied to clipboard
| Challenge: | a conversational system can learn to rank response candidates for a given dialogue context by computing similarity between their vector representations. |
| Approach: | They propose a framework that incorporates augmented dialogue contexts into the learning objective. |
| Outcome: | The proposed framework outperforms existing methods and is more robust to perturbations seen during inference. |
SchAman: Spell-Checking Resources and Benchmark for Endangered Languages from Amazonia (2022.aacl-short)
Copied to clipboard
Arturo Oncevay, Gerardo Cardoso, Carlo Alva, César Lara Ávila, Jovita Vásquez Balarezo, Saúl Escobar Rodríguez, Delio Siticonatzi Camaiteri, Esaú Zumaeta Rojas, Didier López Francis, Juan López Bautista, Nimia Acho Rios, Remigio Zapata Cesareo, Héctor Erasmo Gómez Montoya, Roberto Zariquiey
| Challenge: | Spell-checking as a generation task requires large amount of data, which is not feasible for endangered languages such as the languages spoken in Peru. |
| Approach: | They propose to use augmented misspelling data to train neural spell-checking models for four endangered languages of Peru: Shipibo-Koniba, Asháninka, Yánesha, yine . |
| Outcome: | The proposed model achieves better scores in most of the errors and languages in the four indigenous languages of Peru: Shipibo-Koniba, Asháninka, Yánesha, yine. |
AD-LLM: Benchmarking Large Language Models for Anomaly Detection (2025.findings-acl)
Copied to clipboard
Tiankai Yang, Yi Nian, Li Li, Ruiyao Xu, Yuangang Li, Jiaqi Li, Zhuo Xiao, Xiyang Hu, Ryan A. Rossi, Kaize Ding, Xia Hu, Yue Zhao
| Challenge: | Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. |
| Approach: | They propose a benchmark that evaluates how large language models (LLMs) can help with NLP anomaly detection. |
| Outcome: | The proposed model can perform zero-shot detection without tasks-specific training, data augmentation and model selection, and it can suggest unsupervised AD models. |
Boosting Text Augmentation via Hybrid Instance Filtering Framework (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text augmentation methods generate instances with shifted feature spaces, which leads to a drop in performance on large datasets. |
| Approach: | They propose a hybrid instance-filtering framework that generates instances with shifted feature spaces, which leads to a drop in performance on augmented data. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three classification tasks and nine public datasets. |
CIAug: Equipping Interpolative Augmentation with Curriculum Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Current methods for interpolative data augmentation select samples at random, which might make it difficult for the model to generalize better and converge faster. |
| Approach: | They propose a curriculum-based learning method that leverages the relative position of samples in hyperbolic embedding space as a complexity measure to gradually mix up increasingly difficult and diverse samples along training. |
| Outcome: | The proposed method achieves state-of-the-art results over existing methods on 10 benchmark datasets across 4 languages in text classification and named-entity recognition tasks. |
Target-Aware Data Augmentation for Stance Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for stance detection are not diversified or inconsistent with the given target and label information. |
| Approach: | They propose to augment a text with a conditional masked word prediction task . they propose to replace a target mention with 'target-aware' sentences by replacing a reference word with . |
| Outcome: | The proposed method outperforms existing methods on 11 targets. |
MuMath: Multi-perspective Data Augmentation for Mathematical Reasoning in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) that integrate with external Python interpreters are not able to demonstrate the calculation process, which compromises user-friendliness and understanding of problem-solving steps. |
| Approach: | They propose to use LLaMA-2 to refine LLti-perspective augmentation methods to improve performance. |
| Outcome: | The proposed model achieves 88.3% on GSM8K and 34.5% on MATH. |
Substructure Substitution: Structured Data Augmentation for NLP (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work focuses on word-level manipulation or global sequence-to-sequence style generation. |
| Approach: | They propose a family of data augmentation methods that generalize prior methods by substituting substructures with others having the same label. |
| Outcome: | The proposed methods can be applied to many structured NLP tasks such as part-of-speech tagging and parsing. |
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models (2024.acl-long)
Copied to clipboard
Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, Yongbin Li
| Challenge: | Despite the advances in large language models, they still face difficulties with multi-step reasoning tasks. |
| Approach: | They propose a method that randomly masks certain tokens within the chain of thought to improve model accuracy by 5% over standard supervised fine-tuning. |
| Outcome: | The proposed method improves accuracy and accuracy by 5% over standard fine-tuning with a few codes modified. |
Exploring Representation-level Augmentation for Code Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent data augmentations for code search are at the raw-data level, which requires additional code analysis and training cost. |
| Approach: | They propose a general format of representation-level augmentation that unifies existing methods. |
| Outcome: | The proposed methods can boost the performance of code search models on a large-scale dataset. |
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability (2021.acl-long)
Copied to clipboard
| Challenge: | Using data augmentation to fine-tune pre-trained models with task-specific data has been shown to be ineffective and redundant during fine-timing. |
| Approach: | They propose a data augmentation technique to regularize pre-trained models and encourage them to learn more generalizable features by dropping contiguous spans during training. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the GLUE benchmark and consistently exhibits superior generalization performances on out-of distribution and challenging counterexamples. |
Chain of Thought Prompting Elicits Knowledge Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge augmentation methods require retrieving knowledge from external knowledge sources or developing a reasoner to leverage the logical rules within the external knowledge source. |
| Approach: | They propose a Chain-of-Thought-based method that augments knowledge for deep learning by removing the need for additional knowledge retrieval or knowledge reasoning models. |
| Outcome: | The proposed method outperforms both pure CoT-based methods and the non-augmented method across the majority of 11 publicly available benchmarks for various reasoning tasks. |
Consistency Training with Virtual Adversarial Discrete Perturbation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for regularizing a model are agnostic to the training model and may not be effective for perturbed inputs. |
| Approach: | They propose an augmentation method of adding a discrete noise that would incur the highest divergence between predictions by replacing tokens while keeping original semantics. |
| Outcome: | The proposed method outperforms baselines on semi-supervised text classification tasks and a robustness benchmark. |
Diffusion Based Counterfactual Augmentation for Dual Sentiment Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to augment sentiment models have failed to mitigate spurious association problem inherent in the original data. |
| Approach: | They propose a framework for enhancing sentiment models using an antonymous paradigm and contrastive learning to generate high-quality samples. |
| Outcome: | The proposed framework achieves state-of-the-art performance on four benchmark datasets. |
LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs? (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have compared LLM-based augmentations with established methods, but the results are contradictory. |
| Approach: | They compare the performance of LLM-based augmentation methods with established ones . they found that LLMs are worthy of deployment only when very small number of seeds is used . |
| Outcome: | The proposed methods are worthy of deployment only when very small number of seeds is used. |
Fine-Tuning Encoder-Decoder Models with Contrastive Learning for In-Context Distractor Generation (2025.findings-emnlp)
Copied to clipboard
Elaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Mohammed I. Thanoon, Haojie Zhuang, Behnaz Soltani, Munazza Zaib
| Challenge: | Distractors are used to generate plausible but incorrect options for fill-in-the-blank questions . research studies focus on fine-tuning pre-trained models with data augmentation techniques to generate distractors . |
| Approach: | They propose a model that trains the model to recognize essential semantic features necessary to generate distractors. |
| Outcome: | The proposed model outperforms existing models on two public datasets. |
Soft Contextual Data Augmentation for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for enhancing training data are limited in natural language tasks due to text characteristics. |
| Approach: | They propose a data augmentation method that softly augments a randomly chosen word in a sentence by its contextual mixture of multiple related words. |
| Outcome: | The proposed method outperforms baseline methods on small and large scale machine translation datasets. |
FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation are limited to simple tasks and weak baselines. |
| Approach: | They propose a data augmentation method FlipDA that uses a generative model and a classifier to generate label-flipped data. |
| Outcome: | The proposed method improves many tasks while not negatively affecting the others. |
Target-to-Source Augmentation for Aspect Sentiment Triplet Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment analysis, but data scarcity limits performance of existing methods. |
| Approach: | They propose a target-to-source augmentation approach to alleviate the issue of data scarcity in Aspect Sentiment Triplet Extraction (ASTE) they use fluency and alignment discriminators to provide feedback and use this feedback to optimize the generator. |
| Outcome: | The proposed approach significantly improves the performance of existing methods. |
Document-Level Event-Argument Data Augmentation for Challenging Role Types (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Event Argument Extraction (EAE) are not well-suited to a variety of real-world situations, including long documents and challenging role types. |
| Approach: | They propose two novel methods for generating document-level EAE samples using zero in-domain training data and validate their generalizability. |
| Outcome: | The proposed methods show significant performance increases in low-resource settings. |